Fast Sampling Plane Filtering, Polygon Construction and Merging from Depth Images

Recently, the availability of low cost depth cameras has provided 3D sensing capabilities for mobile robots in the form of dense 3D point clouds, usable for applications like 3D mapping and reconstruction, shape analysis, pose tracking and object recognition. For all the aforementioned applications, processing the raw 3D point cloud in real time and at full frame rates may be infeasible due to the sheer volume of data. Hence, a natural choice is to extract geometric features from the point cloud, and process these features instead of the raw point clouds. The task of geometric feature extraction itself is challenging due to noisy sensing, geometric outliers and real-time constraints. We introduce the Fast Sampling Plane Filtering (FSPF) algorithm to reduce the volume of the 3D point cloud by sampling points from the depth image, and classifying local grouped sets of points using Random Sample Consensus (RANSAC) as belonging to planes in 3D (called the “plane filtered” points) or points that do not correspond to planes (the “outlier” points). The plane filtered points are then converted to a set of convex polygons in 3D which represent the planes detected in the scene. The detected convex polygons are then merged across successive depth images to generate a set of scene polygons. The FSPF and polygon merging algorithms run in real time at full camera frame rates with low CPU requirements: In a real world indoor environment scene, FSPF takes on average 1.39 ms to process a single 640 × 480 depth image, producing 2004 plane filtered 3D points and 70 polygons(average). The polygon merging algorithm takes 1.12 ms to merge the latest frame with the history of obervations. Thus, the algorithms are capable of processing depth images at about 400 frames per second on average. We provide experimental results demonstrating the computational efficiency of the FSPF algorithm, and include results from different indoor scenes.